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The Beginning Teacher Evaluation Study, Phase II, was 
a research project on effective teaching behavior — what teachers do 
that significantly affects what and how pupils learn. The purposes of 
Phase II were to (1) develop an assessment system for measuring 
teacher and student behaviors and other factors which could influence 
each of them and their interrelationships and (2) generate hypotheses 
about the interrelationships among teacher and pupil behaviors and 
related factors. Subjects were 41 second grade and 54 fifth grade 
experienced teachers in eight school districts in California. Pupils' 
reading and mathematics skills were measured. Reading scores were for 
decoding, comprehension, and applications; mathematics scores were 
for computation, concepts, and applications. Students* attitudes, 
aptitudes, cognitive style and expectations were also measured^ The 
teachers were tested on a variety of knowledge and aptitude factors 
including cognitive style. Path analyses and multiple and stepwise 
regression of residual and mean gain scores on teaching performance 
were performed. Results indicate a significant and consistent effect 
of teaching performances on student learning. (RC) 
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Introduction 



Tne major purpose of Phase II of the Beginning Teacher Evaluation 
Study „as to find relations between teaching perf ortnances and achievement 
in reading and n..thematics. The study also assessed the relative 
influence of the antecedents of teaching performances and of learning, 
such antecedents of teaching performances as the teachers' knowledge of 
what was to be taught and of methodological principles and strategies for 
teaching it. the teachers ' attitudes, aptitudes and backgrounds, and the 
characteristics of the teaching environnient were measured to assess their 
influence on teaching performance. Characteristics of pupils such as 
their background, aptitudes, attitudes and expectations were measured to 
assess their influence on learning. Three questions may be answered 
tentatively by analyzing the data gathered: 

1. What is the relative influence of teaching performance on 
learning compared to that of the pupils' previous learning, aptitudes, 
attitudes, and expectations? 

2. How do teaching performances vary as a function of teachers' 
background, knowledge, aptitude, and attitudes? 

3. Which teachiug performances are significantly related to differ- 
ences in pupils' achievement in reading and mathematics? 

This paper presents a partial answer to the last question, partial 
because only some of the data can be presented here. A fuller report on 
the answer to this question may be found in the Final Report of Phase II 
and in its sunnnaries.l 



^' Se^^cp°^p'?"^ documents may be obtained from Educational Testing 
service, Princeton, N.J. 08540. Ask for Final Report, Phase H 
Beginning Teacher Evaluation Study. eporc, rnase LI, 



Methodology 

The Phase II study was a multivariate analysis of teaching perfor- 
mances and learning. The central feature of the design was a fall and 
spriS^ssessment of pupil learning and observations of teaching 
performance during the intervening period. Other teacher and pupil 
variables were also measured in the fall and spring and during the year. 

The basic analytic methodology was to regress pupil spring scores 
on fall scores and then to find the correlates of these residual scores. 
Pupils' reading and mathematics skills were measured. Reading scores 
were for .decoding, comprehension and applications; mathematics scores 
were for computation, concepts and applications. In this paper, teaching 
performances are related only to total reading and mathematics scores. 

Ninety-five teachers in 43 schools in eight school districts and 
their pupils participated in this study. They taught reading or 
mathematics at the second and fifth grade. All data were organized by 
teacher, and each analysis has been made with the data for pupils who 
had both spring and fall scores. 

Each participating teacher was observed on at least two occasions. 
Two different observational systems were used, one a behavioral 
recording system (APPLE), the other, a categorical system (RAMOS). The 
teachers also described on two occasions the content and goals of their 
instruction in reading and mathematics and their teaching strategies 
(Work Diary) .2 ' f'^- 



was°presenteS'"/^'''"' °' ^-^^^ i° -hich this paper 

was presented. See papers by N. Lambert, R. Calfee and P. Elias. 
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Each of these methods for obtaining data on teaching performance was 
reduced to a large number of variables. These sets were further reduced 
to the set presented in Table 1. The original categories were formed 
into groups by collecting different measures of the same phenomena. 
Consider Work Diary 1, for example. This category represents all of the 
original measures of time spent preparing for and conducting instruction. 
Each of its components was weighted in terms of its correlations with 
other components in that category. The correlation matrix of the 
components was used as a multiplicand in the equation 

xR = y 

where x is the teacher's score on the component, R is the correlation 
matrix of the components, and y is the transformed score used to repre- 
sent the teacher's score in the superordinate category. These 
procedures were necessary to reduce the number of teaching performance 
variables so that there were fewer performance variables than teachers, 
and to reduce the multicollinearity among the variables in the original 
data set. 

Two kinds of scores were used, residual scores and mean change 
scores. Analyses were performed by single variables within observation 
systems, by sets of variables by observation system, and by total set. 
Both multiple and stepwise regressions were performed. In the data 
presented here, only the total reading or mathematics score was used. 
This score is the most reliable indicator of pupil performance. 



Results 



Tables 8.16 and 8.22 present the results of the stepwise regressions 
for second-grade reading. Table 8.16 is the results of the regression 
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analysis. using residual scores, Table 8.22 is the results of the analysis 
using mean change scores. Tables 8.19 'and 8.24 are similar tables for ' 
total mathematics scores. Tables 8.32, 8.42. 8.38 and 8.45 present the 
results of comparable analyses for fifth-grade reading and mathematics. 

The results of the stepwise regressions are presented because they 
probably describe patterns of performance more clearly than the other 
analyses. Variables were entered in this analysis in terms of their 
contribution to R.2 Tables 8.25 and 8.46 summarize the results of all 
of the different kinds of analyses. 

Second-Grade Reading (See Tables 8.16 and 8.22) 

Consider the information in Table 8.16. Initial or fall score 
accounts for most of the variance in spring scores. Three tpaching 
performance variables account for 35 percent of the remaining variance. 
Thus initial score plus this pattern of teaching performances account 
for almost all of the variance in spring scores. 

Two of these variables, WD-4 and R-2, are negative predictors. The 
positive predictor, R-3, is a proxy for direct individual instruction. 
WD-4 is proxy for the single teacher classroom. 

A reasonable conclusion is that one teacher carrying most of the 
instruction and trying to do many things is an ineffective combination. 
In contrast, direct individual instruction is an effective strategy. 3 

When mean-change scores are used in the analysis, WD-4 again appears- 
as a negative predictor. Ap-7, teaching the class -as -a-whole is a 
positive predictor. Note that these two variables account for a relatively 
small portion of variance in the mean-change scores. 



3. Other data in this study makes this conclusion even clearer. 
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AP-7 represents a technique for providing some direct instruction 
for all pupUs. The significance of this variable is that it is an 
alternate to independent seatwork in which children spend on-the-average 
50 percent of the instructional time. 

Second-Grade Mathematics' (See Tables 8.19 and 8.24) 

When total mathematics residual scores are used in the analysis, 
again we find that initial scores account for most of the variance in 
spring scores, but six teaching performance variables account for 45 
percent of the remaining variance. Five of these variables are negative 
predictors. 4 

The one positive predictor is the amount of content covered. The 
more mathematics taught, the more is learned, other things being equal 
presumably. 

The pattern of ineffective performances is one in which the teacher 
carries most of the instruction by teaching the class-as-a-whole, probably 
by standing at the frong of the class, explaining and questioning. 

Teaching performance variables account for 63 percent of the variance 
in mean-change scores. Fifteen performance variables produce this 
result, of which ten are negative predictors. Included in these negative 
predictors are the same ones found to be negative in the analysis of 
residual scores. 

The positive predictors include direct instruction by the teacher 
to individual pupils (R-1) , and a social control performance (Ap-13) . 
Two pupil behavior variables, Ap-2_and Ap-3, measures of on-task and 
productive behavior are also positive predictors. 
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Fifth-Grade Reading (see Tables 8.32 and 8.42) 

When residual scores are used in the analysis of fifth-grade total 
reading scores, teaching performance variables account for 31 percent of 
the variance in spring scores after fall scores are partialled out. 
Four variables are negative predictors and three are positive. Two of 
these negative predictors are sijnilar, the use of a variety of instruc- 
tional materials. We think that when this variable appears as a negative 
predictor, it is a proxy for disorganized instruction. - 

One positive predictor, R-4, is independent reading; the other, 
Ap-10, is discussing, explaining, and questioning. Since the emphasis 
in fifth-grade reading is usually on reading comprehension, an effective 
strategy for developing comprehension seems to be independent reading 
during which the teacher discusses the reading materials with the pupils. 

When mean-change scores are used, two performance variables account 
for ten percent of the variance in mean-change scores. One of these, 
Ap-8, Is a measure of the amount of time the teacher spends organizing 
instruction in contrast to instructing. Recall that I suggested when 
discussing the analysis of residual scores, that a variety of materials 
was a proxy for disorganized instruction. 

Ap-10, discussing, explaining and questioning is again a positive 
predictor. Its appearance as a positive predictor suggests that this 
performance is critical in developing skill in reading at the fifth- 
grade. 

Fifth-Grade Mathematics (See Tables 8.38 and 8.45) 

In the analysis of fifth-grade total mathematics scores using 
residual scores, teaching performance variables account for 50 percent 
of the variance in spring scores after fall scores are partialled out. 



Four variables are negative predictors, of which thiee are pupil behavior 
variables. Among the positive predictors. t«o are ways of organizing 
instruction, Ap-6, teaching in groups, and Ap-?, teaching the class-as-, 
a-whole. The third is an interaction strategy, ap-10, discussing, 
explaining and questioning- 

When mean-change scores are used, teaching performance variables 
account for 62 percent of the variance. The positive predictors are 
the amount of content covered (R-2) , keeping pupils on task (Ap-13) , 
teaching the class-as-a-whole (Ap-7) and teaching in groups (Ap-6), and 
discussing, explaining and questioning (Ap-10) . 

The negative predictors are unproductive and off-task behavior, and 
using independent seatwork. Direct instruction by the teacher seems to 
be more necessary than practicing matheraatical operations. 

Summary 

Three major conclusions seem supported by these data: 

1. Teaching performances accouu;.ed for a third to a half of the 
variance in pupil spring scores when their fall scores were partialled 
out. 

2. Teaching performances account for about half of the variance in 
mean-change scores. 

3. Patterns of effective and ineffective performance appeared in 
both subjects and at both grade levels. 

4. These patterns differ by subject and grade level. 

5. At the second-grade direct individual instruction is the core 
of an effective strategy. 
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6. At the fifth-grade direct instruction and interacting with 
pupils in ways which stimulate comprehension responses are the core of 
an effective strategy. 

These differences in effective performances are probably related to 
the differences in what is to be learned at the two grades. At second- ' 
grade discrete responses are being acquired and linked together. 
Therefore, continuous instruction for individuals and monitoring of the 
acquisition process are probably needed. At the fifth-grade cognitive 
processes to be used with varied content are being learned. Teaching 
strategies which stimulate comprehension processes„:are probably required. 



10 
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Table Number Description 

1 Description of Teacher Performance Variables 

^^-^^ Stepwide Regression of Residual Scores on 

Teacher Performance Variables 
Grade 2— Reading 

^'^^ Stepwise Regression of Mean Gain scores on 

Teacher Performance Variables 
Grade 2 — Reading 

^•^^ Stepwise Regression of Reaidual Scores on 

Teacher Performance Variables 
Grade 2 — Mathematics 



8.24 



Stepwise Regression of Mean Gain Scores on 
Teacher Performance Variables 
Grade 2 — Mathematics 



^•25 Summary of Significant Grade 2 Teacher 

Performance Variables Listt^d.by Type of 
Analysis 



^•^2 Stepwise Regression of Residual scores on 

Teacher Performance Variables 
Grade 5— Reading 

^•^2 Stepwise Regression of Mean Gain Scores on 

Teacher Performance Variables 
Grade 5 — Reading 

^•^^ Stepwise Regression of Residual Scores on 

Teacher Performance Variables 
Grade 5 — Mathematics 
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Table Number Description 



8.45 



8.46 



Stepwise Regression of Mean Gain Scores 
on Teacher Performance Variables 
Grade S—Mathematics 

Swnmary of Significant Grade 5 Teacher 
Performance Variables Listed by Type 
of Analysis 



Data cited in this paper were selected from the data gathered in Phase 
II of the Beginning Teacher Evaluation Study conducted by Educational 
Testing Service for the California Commission for Teacher Preparation 
and Licensing and funded by the National Institute of Education, 

copyright 0 1976 by Educational Testing Service. All rights reserved. 
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